1. Field of the Invention
The present invention relates to an image sensing apparatus and a control method therefor, and more particularly, to an image sensing apparatus that senses an object using a solid-state image sensing element such as a CCD, and a control method therefor.
2. Description of the Related Art
In recent years, a wide variety of image sensing apparatuses such as digital cameras and the like, which use a memory card having a solid-state memory element as a recording medium and record and reproduce images sensed with a solid-state image sensing element such as a CCD, have been developed and come into widespread use. At the same time, improved resolution and operating speed with respect to the sensing of still images and moving images in such image sensing apparatuses are sought. Consequently, increasing the frequencies of drive signals for driving the image sensing element and the driving frequencies for analog signal processing circuits, A/D converters, and later-stage digital signal processing circuits that constitute the digital cameras or the like is advancing rapidly.
Moreover, recently, in addition to improving picture quality in terms of providing high picture quality and fine definition, even greater demands are being made for a type of convenience in which almost error-free image sensing in a variety of image sensing scenes is possible. As a result, in order to track fast-moving objects such as in sports scenes, for example, or in an attempt to prevent hand-shake in indoor image sensing under low light, shutter speeds are being increased. Further, to enable image sensing in locations where flash photography is prohibited, such as art museums and aquariums, even greater sensitivity of the image sensing element is sought.
Below, a description is given of an example of a conventional digital camera.
FIG. 13 is a block diagram showing schematically an image sensing part inside a typical digital camera.
In FIG. 13, reference numeral 501 designates an image sensing element such as a CCD or a CMOS sensor (hereinafter “CCD”). Reference numeral 502 designates an image sensing circuit that processes output signals from the CCD 501, and 503 designates an A/D converter for converting processed analog image signals into digital image signals. It should be noted that the area inside the frame designated by reference numeral 500 indicates an analog signal processing part. In addition, reference numeral 504 is a digital signal processor that performs various types of signal processing, for the purpose of recording digitally converted image signals on a recording medium, displaying the signals on a liquid crystal display screen, and so forth. Reference numeral 505 designates an oscillation circuit (OSC1), 506 designates a timing generator (TG), 507 designates a sync signal generator (SSG), and 508 designates an oscillation circuit (OSC2). Reference numeral 509 designates a system controller that includes a CPU for controlling the overall operation of the digital camera.
The OSC1 (505) supplies the operating clocks for the timing generator 506 and the OSC2 (508) supplies the operating clocks for the system controller 509. The timing generator 506 supplies the operating clocks (TGCLK) for the sync signal generator 507. The sync signal generator 507 counts a predetermined number of operating clocks and generates horizontal sync signals (HD) and vertical sync signals (VD) that it then supplies to the timing generator 506. The timing generator 506 synchronizes to the horizontal sync signals (HD) and the vertical sync signals (VD) supplied from the sync signal generator 507 and supplies various drive signals to the CCD 501. In addition, the timing generator 506 supplies sampling clock signals to the image sensing circuit 502, the A/D converter 503, and the digital signal processor 504, respectively. The system controller 509 sets generation or not of the horizontal sync signals (HD) and the vertical sync signals (VD) and their cycles for the sync signal generator 507, and also controls the operation of the digital signal processor 504.
FIG. 14 shows schematically a configuration of the CCD 501 shown in FIG. 13. In FIG. 14, reference numeral 1 designates photoelectric converter elements and 2 designates vertical transfer CCD (VCCD). It should be noted that, of the photoelectric converter elements 1, the photoelectric converter elements 1 indicated by the shading of the leftmost column are light-shielded photoelectric converter elements (shielded part). The other photoelectric converter elements 1 are photoelectric converter elements in an effective pixel area that is not shielded from light. Each photoelectric converter element 1 and each VCCD 2 form a pair, with multiple pairs arranged two-dimensionally to form an image sensing area that senses an image by converting light rays from an object into electrical charges. Reference numeral 4 designates horizontal transfer CCD (HCCD) that transfer electrical charges transferred sequentially from the VCCD 2 in a horizontal direction.
The electrical charges generated at the photoelectric converter elements 1 are transferred to the VCCD 2, after which they are transferred sequentially in a vertical direction toward the HCCD 4, in units of lines in the horizontal direction. Thereafter, the electrical charges are transferred in the horizontal direction by the HCCD 4, converted from charges to voltages by a charge-voltage converter amp 5, and output.
It should be noted that, in reality, there are many more of the VCCD 2, the HCCD 4, the effective pixel area photoelectric converter elements 1 and the light-shielded photoelectric converter elements 1 that comprise the CCD 501 than are shown in FIG. 14. Thus, for example, although in FIG. 14 the light-shielded photoelectric converter elements in the shielded part at a left end of the CCD 501 are shown as a single column, in reality the light-shielded photoelectric converter elements are comprised of multiple columns.
FIG. 15 is a block diagram showing the circuit configuration inside the image sensing circuit 502 in greater detail.
In FIG. 15, the image sensing circuit 502 is comprised of a correlated double sampling (CDS) circuit 600, an amplifier 601, and a clamping circuit 602.
Usually, in a later stage of the CCD sensor, there is a CDS circuit that reduces a reset noise component arising during charge transfer in the CCD. The output of the CCD 501 is comprised of a field through period, which becomes a signal level reference for each pixel during one horizontal transfer cycle, and an image signal period, in which an image signal is output in proportion to the light exposure. The CDS circuit 600 is a noise reduction circuit that obtains a difference between the field through period signal level and the image signal period signal level from the output signals from the CCD 501, and removes a correlative noise component of one pixel cycle from the image signal. The amplifier 601 amplifies image signals output through the CDS circuit 600 to a predetermined signal level to match an input range of the later-stage A/D converter 503, and supplies the amplified signals to the clamping circuit 602. The clamping circuit 602 adjusts a DC voltage level so that charges output from pixels in the shielded part attain a predetermined black reference value. It should be noted that a period in which charges from pixels in the shielded part are output within the image signal period is called an optical black (OB) period.
FIG. 16 shows a timing chart showing the main signals for driving the digital camera shown in FIGS. 13 through 15.
In this example, the OSC1 (505) operating clock frequency is 33.75 MHz and the oscillation circuit 506 operating clock frequency is 27 MHz.
The operating frequency of each pixel output from the CCD 501 is determined by CCD drive signals generated by the timing generator 506, and is generated from the same 33.75 MHz frequency as the OSC1 (505) operating clock. In other words, one pixel period of CCD output signal at this time is 29.6 ns, (=1/33.75 MHz). The field through period and the image signal period described above are contained in one period of the operating clock.
Further, the timing generator 506 generates, in sync with the CCD drive signal, a sample-and-hold S/H pulse (SH1) that samples and holds the signal level of the field through period at each pixel and a sample-and-hold S/H pulse (SH2) that samples and holds the signal level of the image signal period.
An increase in the speed of the driving frequency of the image sensing elements in the image sensing apparatus can be one major cause of deterioration of an S/N ratio of the image signals. Leakage of unnecessary clock signals into analog image signals occurs particularly easily inside digital image sensing apparatuses, which are driven by a plurality of operating clock signals and in which analog signals and digital signals are mixed together. Unnecessary clock signals leaking into the analog signals are superimposed on the generated image at a fixed pitch as interference clock noise. Moreover, since the signals are of fixed pitch, the leaking clock signals are often noticeable even though they are at a smaller level than the level of random noise such as the thermal noise of the CCD sensor and circuits.
In addition, this type of deterioration in the S/N ratio of the image signal surfaces all the more readily the higher the sensitivity settings of the image sensing apparatus and the greater the degree of amplification of the image signals of the image sensing circuit.
For example, in the configuration shown in FIG. 13, as the system operating speed is increased and the image sensing signal driving frequency is increased, the S/H pulses (SH1, SH2) and the A/D converter 503 sampling clock (ADCLK) have also increased in speed. As a result, it becomes increasingly difficult to avoid these clock noise components (system clock components) leaking into the image signals of the analog signal processing area 500 by adjusting the timing.
Here, a description is given of a case in which the output of the CCD 501 is subjected to sampling by the A/D converter 503 sampling clock (ADCLK) and the S/H pulses (SH1, SH2) with a pixel clock frequency of 33.75 MHz. If the above-described system clock components (27 MHz) leak into image signals in the analog signal processing area 500, then 6.75 MHz of cyclical noise, which is the difference between the frequency components (=33.75−27 MHz), remains in the post-sampling image data as a result. This is one fifth the frequency of the 33.75 MHz CCD 501 drive pulse, or in other words fixed pitch noise of 5 pixel cycles. The fixed pitch noise created by this pulse interference, although it depends also on the pitch size, is easily distinguishable compared to the random noise such as the thermal noise of the CCD sensor and image sensing circuit, and therefore tends to be especially noticeable.
In the case of the CCD sensor, a rough breakdown of a single horizontal (1 H) period is as shown in FIG. 17. Specifically, the horizontal period is composed of a blanking period, in which HCCD 4 transfer drive pulses H1, H2 are stopped, and a pixel readout period in which HCCD 4 transfer drive pulses H1, H2 are driven (OB period+effective pixel period).
The fixed pitch noise superimposed one-dimensionally on the image signals, with respect to a two-dimensional image rendered horizontally and vertically by the CCD area sensor or the like, changes appearance depending on the number of pixel clocks that comprise a single horizontal period.
In the case of fixed pitch noise of 5 pixel cycles, as shown in FIGS. 18A through 18E, the noise pattern formed has five variations according to cosets of 5 according to the number of pixel clocks that comprise a single horizontal period. As can be seen from FIGS. 18A through 18E, although there is no change in the noise pitch in the horizontal direction, the way in which the noise shows up changes somewhat because the angle of the noise pattern on the rendered two-dimensional image changes.
However, in an interline-type solid-state CCD, there is an output method in which the electrical charges of all the pixels are read out in n portions, that is, a single frame is divided into n fields and output.
FIG. 19 shows a timing chart illustrating the timing of an image sensing operation when outputting the charges of all the pixels (a single frame) in three fields, in an image sensing apparatus using the interline-type CCD described above.
In FIG. 19, VD is a vertical sync signal and HD is a horizontal sync signal. In a mechanical shutter, exposure is controlled by mechanically opening and closing the mechanical shutter. In an electronic shutter, the exposure is controlled by applying a pulse to the CCD 501 substrate potential and pulling the pixel charges toward the substrate (resetting). A time period from completion of resetting of the pixel charges by the electronic shutter to closing of the mechanical shutter is an exposure period. In addition, a time period from the completion of the resetting of the pixel charges by the electronic shutter to output of the pixel charges of the photoelectric converter elements 1 to the VCCD 2 is a charge accumulation period.
The CCD 501 output operation, after completion of exposure by the mechanical shutter, is divided three times by a plurality of vertical transfer pulses, not shown, applied to the CCD 501 from the timing generator 506, with only charges of pixels of certain horizontal lines, each different, output to the VCCD 2. Therefore, the readout of a single frame constructed of N lines would be output as follows:
(1) In an initial vertical sync period, output charges from pixels of lines corresponding to lines 1, 4, 7 . . . N−2 as a first field;
(2) in a succeeding vertical sync period, output charges from pixels of lines corresponding to lines 2, 5, 8 . . . N−1 as a second field; and
(3) in a another succeeding vertical sync period, output charges from pixels of lines corresponding to lines 3, 6, 9 . . . N as a third field.
In an image sensing apparatus using an interline-type solid-state image sensing element like that described above, the above-described fixed pitch noise of 5 pixel cycles superimposed one-dimensionally on the image signals develops into noise patterns like those shown in FIGS. 20A and 20B on the two-dimensional image during readout of all the pixels. The noise patterns shown in FIGS. 20A and 20B are combinations of the noise patterns shown in FIG. 18B, combining three fields to make up a single frame. In both FIG. 20A and FIG. 20B cyclical features of the noise patterns of each of the fields are emphasized, resulting in heightened visibility.
If the phase relation can be accurately controlled so that cyclic noise pixels on adjacent lines between the fields are skillfully dispersed, then it would seem to be possible to wipe out the cyclical features of the noise patterns of each of the fields and construct a frame image in which the noise pattern is difficult to identify.
However, in most cases, the OSC1 (505) operating clock (33.75 MHz) and the OSC2 (508) operating clock (27 MHz) shown in FIG. 13 are configured with mutually independent oscillation circuits. In the case of a free run, the phases of the noise patterns of the fields move depending on accuracy and temperature drift of the respective oscillation circuits. Consequently, when producing one frame image from three field images, the noise patterns are sometimes easily noticeable and sometimes hardly noticeable, and this wide variation poses a problem.
It is also possible to generate the OSC1 (505) operating clock (33.75 MHz) from, for example, the OSC2 (508) operating clock (27 MHz) with a method that uses a phase locked loop (PLL) circuit or the like. In this case, the noise pattern and phase relation between fields can be determined accurately, and therefore the frame image noise pattern can be controlled. However, in such case the OSC2 (508) operating clock (27 MHz) must be taken through the PLL circuit to the timing generator 506 and near the analog signal processing area 500 that is so susceptible to noise, thereby increasing the risk of substantial leakage of the system operating clock (27 MHz) into the analog image signals.
On the other hand, when using a solid-state image sensing element such as a CCD, ordinarily dark noise correction is carried out. Dark noise correction is a calculation process carried out using main image data, which is read out after carrying out electrical charge accumulation in a state in which the image sensing elements are exposed, and dark image data, which is similarly read out after carrying out electrical charge accumulation but in a state in which the image sensing elements are not exposed. Dark noise correction can correct the sensed image data with regard to picture quality degradation due to dark current noise generated by the image sensing elements, pixel loss due to minute scratches unique to image sensing elements, or the like to achieve high-quality image sensing.
In particular, because dark current noise increases with charge accumulation period and image sensing element temperature increase, dark noise correction can obtain substantial picture quality improvement when conducting long time exposure or high-temperature exposure.
In an image sensing apparatus that carries out such dark noise correction processing, the fixed pitch noise of 5 pixel cycles superimposed on the image signals as described above develops into noise patterns like those shown in FIGS. 21A-21E on the image after dark noise correction processing.
The noise patterns shown in FIGS. 21A-21E are examples of dark noise correction processing (in this case subtraction) carried out on both main image data and dark image data, when one horizontal period is composed of 5N+1 pixel clocks as shown in FIG. 18B.
As shown in FIGS. 21A-21E, five noise patterns of from FIG. 21A to FIG. 21E are generated in the image after dark noise correction processing due to a phase difference between the main image data and the dark image data.
FIGS. 21A-21E together create an approximate distribution state in which the cyclical noise level before the dark noise correction processing is 2 at the center pixels and 1 at the adjacent pixels, and show schematically a state in which the noise level after dark noise correction processing changes due to the noise pattern phase difference with the dark image.
For the five noise patterns, the way in which the noise shows up may be divided into three cases, specifically:
(1) cases in which the noise is less noticeable compared to the main image (phase difference 1, FIG. 21A and FIG. 21C);
(2) cases in which the noise is very noticeable compared to the main image (phase difference 2, FIG. 21D and FIG. 21E); and
(3) cases in which noise is not at all noticeable compared to the main image (phase difference 0, FIG. 21B).
If the phase relation can be accurately controlled so that cyclic noise pixels on the main image and the dark image are skillfully made to cancel each other out, then it would seem to be possible ultimately to produce an image that is free of cyclical noise.
However, in most cases, the OSC1 (505) operating clock (33.75 MHz) and the OSC2 (508) operating clock (27 MHz) shown in FIG. 13 are configured with mutually independent oscillation circuits. In the case of a free run, the phases of the noise patterns of the main image and the dark image move depending on the accuracy and the temperature drift of the respective oscillation circuits, and it is very difficult to control the phase relation accurately.
It is also possible to generate the OSC1 (505) operating clock (33.75 MHz) from, for example, the OSC2 (508) operating clock (27 MHz) with a method that uses a phase locked loop (PLL) circuit or the like. In this case, the noise pattern and phase relation between frame fields can be determined accurately, and therefore the noise pattern after dark noise correction processing can be controlled. However, in such case the OSC2 (508) operating clock (27 MHz) must be taken through the PLL circuit to the timing generator 506 and near the analog signal processing area 500 that is so susceptible to noise, thereby increasing the risk of substantial leakage of the system operating clock (27 MHz) into the analog image signals.
In Japanese Patent Application Laid-Open 2001-285726 a technology for reducing unneeded beat noise generated when applying frequency dispersion means to analog signal processing in an image sensing apparatus is disclosed. According to this technology, superimposing beat noise that responds cyclically to frequency dispersion onto the image signals using the frequency dispersion means is suggested. Then, in a horizontal transfer blank period of the image sensing elements the phase of the frequency dispersion part is randomly reset to achieve a reduction in beat noise.
However, in a digital camera that does not have a frequency dispersion means like that shown in FIG. 13, unnecessary cyclical noise generated outside the frequency dispersion means cannot be reduced with the method of Japanese Patent Application Laid-Open 2001-285726.